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Abstract 

Over a decade ago, Barr and Tagg (1995) declared that a shift had occurred in higher 
education from an instruction paradigm to a learning paradigm. A central element in this new 
paradigm is learner-centered assessment. While a growing body of literature suggests that this 
approach to assessment is a best practice in higher education pedagogy, it is still unclear whether 
faculty members have embraced it fully. Using data from the 1993 and 2004 National Study of 
Postsecondary Faculty, this study examines the extent to which faculty members employ 
learner-centered assessments in postsecondary classrooms and compares use of select 
assessments from 1993 to 2004. Findings show stable to increased use of some assessment 
techniques over the decade as well as differences by faculty gender, discipline, and institution 
type. Implications for faculty members, student learning, and institutional policy are discussed. 
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Introduction 

Refonning instruction in postsecondary education continues to be a salient issue among 
educators, leaders in professional organizations, accrediting bodies, and elected officials, 
especially as it pertains to the assessment of student learning (e.g., Ewell, 2002; Glenn, 2011; 
Kinzie, 2010). Assessment has been a long-standing issue and an important dimension of quality 
instruction, however, its meaning and use have evolved significantly over the past two decades. 

A traditional view of assessment defines its primary role as evaluating a student’s 
comprehension of factual knowledge. A more contemporary definition, which is growing in 
popularity, sees assessment as an activity designed primarily to foster student learning. This 
more recent definition has its roots in a movement to make higher education more learner- 
centered, and assessment practices aimed at this goal are often labeled learner-centered 
assessment techniques. 

It is clear from a growing body of literature that learner-centered assessment is now 
considered a highly-valued practice in higher education pedagogy. Y et, it is unclear to what 
extent today’s faculty are employing learner-centered assessment techniques and how that 
compares to the early 1990s, prior to extensive discussion on learner-centered education. 

Previous studies using US national data do not extend into the new millennium, and no studies 
have examined data from the most up-to-date study of postsecondary faculty, the 2004 National 
Study of Postsecondary Faculty (NSOPF: 2004). This study examines responses from the 1993 
and 2004 administrations of the National Study of Postsecondary Faculty to better understand 
the use of learner-centered assessment techniques in today’s postsecondary classrooms and to 
detennine changes over the -decade. This study departs from previous studies on this topic in its 
application of hierarchical linear modeling. An advantage over single level regression, multilevel 
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analysis enables us to examine the effects of individual characteristics on learner-centered 
assessment practice, nested within the effects of institutional characteristics. This technique 
provides a more precise examination of the contribution of individual and institutional 
characteristics on learner-centered assessment. 

Conceptual Framework 

The conceptual framework for this study is guided by two areas of literature, institutional 
theory and assessment practices in higher education. Institutional theory is most concerned with 
effects that have causes rooted in institutions (Powell & DiMaggio, 1991). According to 
Jepperson (1991), an institution “represents a social order or pattern that has attained a certain 
state or property” (p. 145). When this pattern is reproduced, regulated, and preserved through 
socially constructed rewards and sanctions, it is said to be institutionalized. Institutional theory 
also asserts that organizations are characterized by rules and requirements to which organizations 
must conform in order to be successful (Scott & Meyer, 1983). Assessment efforts endeavored 
by institution officials are an example of requirements that help an institution become a 
successful organization. In addition, institutional theory broadly argues that a need for legitimacy 
drives organizational behavior, which, in turn, promotes isomorphism among organizations 
contained in the same field, such as higher education (DiMaggio & Powell, 1983). Organizations 
that successfully incorporate environmentally-legitimated elements are more likely to succeed, 
grow, and survive over time than those that fail to confonn. From this perspective, universities 
and colleges can be viewed as organizations seeking to conform with regulations and 
requirements mandated by external constituents. One area of regulation of particular importance 
to this study is accountability. Over the past decade, policymakers, accrediting bodies, and 
elected officials have continued calls for improved mechanism to foster and measure student 
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learning. The proceeding literature review demonstrates the extent to which these calls have been 
institutionalized in the field of higher education. Institutional theory would expect universities 
and colleges that are seeking legitimacy to align organizational structures, personnel, and 
instructional processes with requirements and regulations around student learning. Along this 
line of thought, one would predict an increase in activities, such as learner-centered assessment, 
over time. 

Due to its value in the teaching-learning process and because it assists with accountability 
needs, the assessment of teaching and learning in postsecondary education has become 
inextricably incorporated into institutional policies and practices. Institution-wide and learning 
outcomes assessment is now considered imperative for institutional improvement and generally 
required by regional and discipline accreditation agencies. Because internal and external agents 
are looking for practices related to and evidence of learning outcomes, faculty are strongly 
encouraged to assimilate instructional and assessment techniques that provide evidence of 
learning. As the emphasis on assessment grows, more faculty members within and across 
institutions are aware of specific techniques that can achieve goals for learning. Generally, there 
are three different levels of assessment in colleges and universities: institution-, program-, and 
course-level. While a learner-centered assessment process encompasses all levels, the present 
research is exclusively focused on course-level assessment. Assessment techniques at this level 
are fundamentally similar. They: 1) derive from specified learning outcomes; 2) compel students 
to apply the knowledge and skills — their learning, in other words — to discipline-specific issues 
and problems; 3) include meaningful, constructive feedback from faculty; and 4) occur at 
intentional points during the learning process (Angelo, 1999; Huba & Freed, 2000; Mezeske & 
Mezeske, 2007). Examples of learner-centered assessment activities include: multiple drafts of 
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written work in which faculty provide constructive and progressive feedback; oral presentations 
by students on discipline-specific problems; student evaluations of each other's work; group and 
team projects producing a joint product related to specified learning outcomes; and service 
learning assignments requiring interactions with the community or business/industry. All of these 
activities promote learner-centered education and can provide faculty with evidence regarding 
how effectively students can construct their knowledge and skills to address course-related 
learning. 

Literature Review 

Over a decade ago, Barr and Tagg (1995) declared that a shift had occurred in higher 
education from an instruction paradigm to a learning paradigm. In the learning paradigm, faculty 
focus less on transferring factual knowledge to students and more on creating a learning 
environment that empowers students to construct knowledge for themselves and apply it to 
complex problems. The learning paradigm positions the learner, rather than the instructor, at the 
center of undergraduate education, and, for this reason, many refer to this paradigm as learner- 
centered education. 

Since Barr’s and Tagg’s declaration, many theorists and leaders in higher education have 
endorsed learner-centered education and extended the conversation about its utility in American 
colleges and universities. In 1998, for example, the American Association for Higher Education 
(AAHE), College Student Educators International (ACPA), and Student Affairs Administrators 
in Higher Education (NASPA) appointed the Joint Task Force on Student Learning. The Task 
Force produced a report that summoned those invested in higher education to adopt key 
principles that encourage learner-centered education and work collaboratively to implement 
practices that deepen the learning experience on college campuses. Along similar lines, authors 
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of Learning Reconsidered argued for the redirection of all of higher education’s resources 
toward the goals of transformative education, a “holistic process of learning that places the 
student at the center of the learning experience” (NASPA & ACPA, pg. 1, 2004). More recently, 
the Association of American Colleges and Universities (AACU) sponsored several reports on 
college learning that pointed up the important role learner-centered pedagogy plays in achieving 
the “essential learning outcomes” of liberal education (National Leadership Council for Liberal 
Education and America’s Promise [LEAP], Crutcher, Obrien, Corrigan, & Schneider, 2007; Kuh, 
2008). 

Across this body of literature, advocates describe assessment as a central element in 
learner-centered education. Assessment in this approach, however, departs from its traditional 
role as an activity used primarily to evaluate a student’s comprehension of factual knowledge. 
Rather, assessment becomes an activity that fosters student learning. Huba and Freed (2000) 
define assessment in a learner-centered approach as an, “activity, assigned by the professor, who 
yields comprehensive infonnation for analyzing, discussing, and judging a learner’s perfonnance 
on valued abilities and skills” (p. 12). They labeled this type of assessment, “learner-centered 
assessment,” and outlined ways in which it reinforces the attributes of learner-centered 
education. We distilled this outline into a more succinct bullet-point list and provided it below to 
aid the reader. According to Huba and Freed, learner-centered assessment: 

• promotes high expectations in the learning environment; 

• respects diverse talents and learning styles; 

• enhances the early years of undergraduate study; 

• promotes coherence in learning by providing data to direct curriculum development and 
revision processes; 
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• synthesizes experiences, fosters ongoing practice of skills, and integrates education and 
experience; 

• involves students more deeply in learning and promotes adequate time on task during a 
program of study; 

• provides prompt feedback to students; 

• fosters collaboration with peers and faculty; and 

• results in increased student-faculty contact. 

This list sheds light on the characteristics found in assessment techniques or methods that 
align with this approach to assessment. At the same time, it demonstrates the value of learner- 
centered assessment in shifting the focus of undergraduate education to student learning. When 
faculty place the student at the center of undergraduate educational, the role of assessment 
transforms from purely summative to mostly formative and becomes another pedagogical tool 
employed to involve students more actively in the construction of their learning experience. 
Recent Evidence 

Learner-centered assessment has been the subject of much scholarship in recent years. 
Research in this area has covered different postsecondary sectors (Boyer, Butner, & Smith, 

2007), general education and various disciplines (Goubeaud, 2010; Paradis & Dexter, 2007; 
Palomba, 2002; Yanowitz & Hahs-Vaughn, 2007), professional and graduate programs (Gerdy, 
2002; Goubeaud & Yan, 2004; Candela, Dailey, & Benzel-Lindley, 2006; Boaten, Bass, Blaszak, 
& Farrar, 2009 and even international higher education (Fook & Sidhu, 2010). Taken together, 
studies reveal a growing interest in reforming assessment in higher education to be more learner- 
centered. These studies do not, however, provide convincing evidence that reform has actually 
occurred. Although some scholars extol the virtues of learner-centered assessment (primarily in 
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conjunction with problem-based learning, for example Duch, Groh & Allen, 2001; Wood, 2008; 
Wood, 2004), Yanowitz and Hahs-Vaughn (2007), for instance, drew data from the National 
Study of Postsecondary Faculty 1993 (NSOPF:1993) and 1999 (NSOPF: 1999) data sets to 
examine assessment activity among faculty in science disciplines. They found that science 
faculty were less likely to use learner-centered assessment practices than non-science faculty and 
that the former did not increase their use of these practices between the two national studies, 
despite science educators appealing for such an increase throughout the 1990s. 

Goubeaud and Yan (2004) point to similar appeals made by teacher educators over the last 
two decades. They used NSOPF:93 data to explore assessment practices among teacher 
education faculty. Findings from their study suggested that teacher educators were more likely to 
employ learner-centered assessment methods, such as research papers and essay exams, than 
other faculty, who relied on more traditional assessment practices, such as multiple choice 
exams. Several scholars have advocated the need for adoption of learner-centered assessment. 
For example, Paradis and Dexter (2007) illustrated the benefits of learner-centered education to 
geography faculty and students and described in detail a field analysis course that exemplified its 
tenets. Similarly, Candela, et al (2006) discussed the benefits to nursing faculty and students and 
presented a model depicting the role assessment plays in establishing a learner-centered nursing 
curriculum. Gerdy (2002) argued for the implementation of learner-centered assessment in legal 
research instruction and provided examples of exemplary assessment activities that could be 
employed by law professors. 

A number of studies have examined the effects of faculty demographics on teacher 
behaviors or pedagogical approaches. Nelson Laird, Garver, and Niskode-Dossett (2011) found 
female faculty use more active learning techniques than male peers Bennett (1982), Centra & 
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Gaubatz (2000) and Statham, Richardson, & Cook (1991) report that female faculty are more 
student-oriented and less authoritative than male faculty, have more class discussions, do less 
formal lecturing, and are more available outside class. In general, these characteristics allow 
students to feel more comfortable in class, and may encourage participation. Recent studies have 
also examined instructor effectiveness by time and tenure status. Gappa and Leslie (1993; 1997) 
believe part-time faculty are generally satisfied with their jobs and are important contributors to 
postsecondary education. Although Umbach (2008) found that part-time faculty use active 
learning less than full-time faculty, Leslie & Gappa (2002) report that part-time faculty, are a 
“stable component of the faculty workforce and have considerable teaching experience” (p.61). 
Leslie and Gappa agree with other reports (Bolge, 1995; Cohen & Brawer, 1998; Grubb, 1999) 
that find no significant differences in quality of instruction between full-time and part-time 
faculty. 

In recent years, most part-time and increasingly more full-time instructional staff 
members are hired in positions that do not offer tenure. Gappa, Austin and Trice (2007) report 
that in 2004 only 27 percent of all new faculty appointment and 56 percent of all full-time 
appointments were tenure-track positions. According to Kezar & Sam (2010), findings are mixed 
on student success (or lack of success) based on faculty member time and tenure status. Several 
researchers found that enrollment in more lower division classes with part-time faculty (who 
were also non-tenure track) were less likely to return for the second year (Bettinger & Long, 
2005), transferring to baccalaureate institution (Eagen & Jaeger, 2009), and/or graduating 
(Ehrenberg & Zhang, 2005), yet Waller and Davis (2009) found no significant differences in 
enrollment growth or retention for systems in one state by tenure status. Authors such as Kezar 
and Sam recognize the complexities of contingent faculty, including the implications for faculty 
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members themselves, students, and the overall organization of higher education, and while a 
number of studies have addressed the effects of contingent faculty, there is much more study 
needed. 

Along with faculty member characteristics, scholars have also examined instructional 
differences by academic discipline. Some scholars purport that the teaching-learning process is 
different across disciplines (e.g., Becher & Trowler, 2001; Hannan & Silver, 2000; Healy & 
Jenkins, 2003; Neumann, 2001; Young, 2010). Neumann & Becher (2002) believe that 
epistemological differences affect teaching and learning. The ‘hard’ sciences tend to focus on 
experimentation, facts, and quantifiable data, whereas the ‘soft’ sciences leave room for context- 
specific interpretations of a given phenomenon. Discreet and intertwined facets of knowledge 
may lend themselves to different forms of assessment. The former may best be measured by a 
multiple choice exam that reinforces memorization of facts, while the latter may be served well 
by essay exams that require students to demonstrate argumentation and theoretical synthesis. 

Statement of Problem 

It is clear from this body of research that learner-centered assessment is now considered a 
valued practice in higher education pedagogy. As the paradigm shifted from one of instruction to 
one of learning, assessment transfonned into an activity for promoting — rather than simply 
measuring — learning. Based on the growing volume of literature on the subject, it appears that 
the trend towards learner-centered assessment has gained momentum in recent years, permeating 
academic discourse in most traditional disciplines. Yet, it is still unclear how much this trend has 
actually influenced faculty assessment practices in postsecondary classrooms in the new 
millennium. While some faculty members seem to have embraced learner-centered assessment 
techniques, others appear more resistant. In addition, previous studies have analyzed national 
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data only through the early 1990s; researchers have yet to consider data from the most up-to-date 
federal study of postsecondary faculty, NSOPF: 2004. These factors indicate a need for further 
investigation into faculty assessment practices using the latest national data and provide the 
opportunity to compare use of touted assessment practices over the decade from 1993 to 2004. 
The purpose of this study is to address these needs by examining the extent to which today’s 
higher education faculty incorporate learner-centered assessment methods into instruction and 
compare frequency rates from 1993 to 2004. The following research questions guided this study: 

1 . How regularly do college and university faculty members employ learner-centered 

assessment techniques in their classes? 

2. Does assessment practice differ by Carnegie classification and faculty tenure status, gender, 
rank, discipline, and position type? 

3. Are there differences in the frequencies of use between 1993 and 2004? 

4. From 1993 and 2004 faculty responses, which individual and institutional factors help explain 
variation in assessment practice among higher education faculty? 

Methodology 

Data 

This study examined faculty response data from the National Study of Postsecondary 
Faculty surveys administered in 1993 and 2004 (NSOPF:93 and NSOPF:04). Because the focus 
of this study is on assessment practices in undergraduate education, only a subset of institutions 
and respondents were included. The final dataset included only those universities and colleges 
defined in each dataset as Associate-granting or above. The 1993 categories were “Doctoral- 
granting, Comprehensive colleges, Liberal Arts, and Associates; the 2004 categories were 
Carnegie 2000 “Doctoral granting,” “Master’s granting,” “Bachelor’s granting,” and 
“Associate’s granting.” Institutions defined as “other” were excluded. In addition, the final data 
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included only instructors who had faculty status, identified teaching as a principal activity, spent 
at least 50% of their time on instructional activities, and taught at least some undergraduate 
courses for credit. Otherwise, the subset included all full-time and part-time faculty at all ranks 
and tenure statuses. The 1993 final dataset included responses from approximately 15,000 
faculty members in nearly 700 colleges and universities; the 2004 dataset included responses 
from approximately 16,000 faculty in over 550 colleges and universities. Using the weight 
provided by NCES, the samples represent approximately one million faculty members in two- 
year and four-year American universities and colleges. 

All analyses described below included the faculty and institution weights provided in the 
NSOPF 1993 and 2004 data sets, as well as an additional weight calculated to correct for 
possible oversampling. We calculated this second weight by dividing raw faculty weight by its 
mean, thus creating a relative faculty weight. In a traditional one-level regression analysis, the 
effects of clustered samples would need to be addressed statistically; however, multilevel 
analysis naturally accounts for clustering by producing estimates for within and between-group 
variances. Therefore additional statistical manipulation, such as accounting for design effects, is 
not necessary (Hahs-Vaughn, 2006; Thomas & Heck, 2001). 

Variables 

As discussed above, learner centered assessment includes a variety of activities that seek 
to involve students more deeply in the learning process, integrate education and experience, 
include feedback from faculty and peers, and address discipline-specific issues and problems 
(Huba & Freed, 2000). In the NSOPF:93 survey, respondents were ask to indicate how often they 
used nine assessment techniques (not used at all, used in some classes, used in all classes). 
Similarly, NSOPF:04 respondents were asked to indicate how often they used ten pre-determined 
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assessment techniques. From the 2004 data, we identified five assessment techniques that were 
clearly consistent with Huba’s and Freed’s operational definition of learner-centered assessment 
to serve as dependent variables. The five techniques were multiple drafts of written work, oral 
presentations, group projects, student evaluations of each other’s’ work, and service learning/co- 
op interactions with business. Of these five, three were included in the 1993 survey (multiple 
drafts of students’ work, oral presentations, student evaluations of each other’s work). No other 
items in the 1993 data were consistent with Huba and Freed’s definition of learner-centered 
assessment. Following previous analyses of 1988 and 1993 NSOPF data (Yanowitz & Hahs- 
Vaughn, 2007) we also created a composite score representing a participant’s overall use of 
learner-centered assessment techniques. This score was created by adding together a participant’s 
scores on each of the three (1993) and five (2004) learner-centered techniques. This composite 
score was used as the dependent variable in the two-level hierarchical models presented in the 
results. 

Independent variables were selected to explore the effects of faculty and institutional 
characteristics on assessment practice. Variables representing faculty characteristics included 
measures for rank, tenure status, gender, discipline, and time status. Rank consisted of full 
professor, associate professor, assistant profess, instructor, lecturer, and an “other” category. The 
variable for tenure status included tenured, on the tenure track but not tenured, not on tenure 
track, and no tenure system at institution. The discipline variable was derived from Biglan’s 
classification of academic disciplines, which included eight categories (1973a; 1973b). Appendix 
A provides a table to show how NSOPF teaching fields were grouped in Biglan categories. 

Because institutional characteristics have been shown to relate to instructional practices 
(Ewell, 1988; Peterson & Vaughan, 2002), we also included several institutional variables in the 
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analysis. The percent of an institution’s total expenditures devoted to instruction was calculated 
to represent the value of instruction to an institution (proportion expenditures for instruction 
divided by total expenditures). The ratio of students to faculty was used to signify the quality of 
an institution’s classroom learning environment. Institutional mission, organization, and scope 
were defined through variables for institutional control (private vs. public) and Carnegie 
classification (as described above in the methods data section). 

Analysis 

To examine research questions (RQ) 1-3, descriptive statistics were completed for 
selected dependent and independent variables (see Tables 1 and 2). Descriptive statistics reveal 
how regularly faculty members employed learner-centered assessment techniques in their classes 
(RQ1 and RQ3). The second stage in the analysis examined whether assessment practices 
differed by Carnegie classification and faculty tenure status, gender, rank, discipline, and 
position type (RQ2 and RQ3). Table 2 shows the proportion of faculty using learning-centered 
assessment in at least some classes (not used at all=l; used in classes some of the time=2; used in 
classes all of the time=3) by important demographic characteristics and discipline. Chi-square 
tests were used to determine significant associations between the categorical independent 
variables and selected dependent variables. 

Guided by our theoretical model of assessment and results seen in the descriptive 
characteristics analysis, we constructed two hierarchical linear models to explain variation in 
assessment practices among higher education faculty (RQ4). A two-level model nested faculty 
members at level one, within institutions at level two. Multilevel analyses allow us to account 
more precisely for error in variance while controlling for differences among individuals nested 
within institutional environments. Tables 3 and 4 show descriptive statistics for variables 
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included in each multilevel model (one for 1993 and one for 2004), and Table 5 presents the 
results from the multilevel analyses. 

Results 

As shown in Table 1 for both 1993 and 2004 data, male and full-time faculty members 
are in the majority, comprising about 60% of the sample (unless specified, use of the term faculty 
member includes all rank levels). In 2004, however, the number of female faculty with 
instructional duties increased by about 3% in 2004 and the number of part-time faculty increased 
by about 8% over the decade. In terms of rank, instructors account for about 1/3 of the total 
group, but we see shifts in the distribution of faculty by rank over the decade. Compared to 1993, 
there were fewer full and associate professors in 2004, and significantly more faculty who teach 
with a title other than professor, instructor, or lecturer. Notable changes in tenure status are also 
noted from 1993 to 2004. The number of tenured faculty dropped by about 25% and as shown, 
the number of 2004 faculty who teach in a non-tenure-track line more than doubled since 1993. 

Faculty distribution by discipline is also shown in Table 1. While the number of faculty 
in Hard Pure Life, Hard Pure Non-Life and Soft Pure Life stayed relatively constant, the number 
of faculty in the Hard Applied Life discipline dropped while Soft Applied Life and Soft Pre Non- 
Life increased. Overall, about 2/3 of the respondents were teaching at public institutions, and the 
majority were doing so in Doctoral- and Master’s, and Associate-granting institutions. 

Table 1 also shows the percentage of faculty using the learner-centered assessment 
techniques. In general across both time points, over half of the faculty said they used these 
assessment techniques in at least some of their classes. Of the three measures across both time 
points, oral presentations by students was used the most frequently. Comparing use in 2004 to 
1993, we see some interesting changes. In 2004, faculty respondents said they used multiple 
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drafts of assignments more often, and three times the number of faculty in 2004 used multiple 
drafts in all classes compared to those in 1993. Compared to 1993, faculty respondents in 2004 
said they used more oral presentations by students, and used more opportunities to have students 
evaluate each other’s work. 

Although comparable items for 1993 are not available, Table 1 shows the percentage of 
2004 faculty who use group/team projects and service learning, community, or co-op 
experiences. Responses show that over half of the faculty use group/team projects and just over 
35% of the faculty use service-learning/community/co-op. Of the five learner-centered 
techniques examined in this study, service-learning/co-op/community was used the least. 

(Insert Table 1 about here) 

Table 2 presents the proportion of faculty using learner-centered assessments by select 
demographic and institutional characteristics. Generally across the assessment techniques at both 
time points, female and full-time faculty reported using these techniques significantly more than 
male and part-time faculty, respectively. Compared to faculty in other ranks, full professors 
reported using all five techniques less; generally, associate and assistant professors indicated 
using these techniques most frequently. Tenured faculty reported lower levels of use than tenure- 
track faculty and those not in a tenure-track position. Differences in use by discipline were also 
noted. In general, faculty members in soft disciplines reported higher use than peers in hard 
disciplines. Examining use by Carnegie group revealed that faculty members in 
bachelor ’s/liberal arts and master’s/comprehensive institutions employed these assessment 
techniques to measure student learning most frequently. 

(Insert Table 2 about here) 
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Table 2 also allows us to examine changes from 1993 to 2004 for three of the learner- 
centered assessment techniques. Compared to responses form 1993, faculty in the 2004 study 
report use of multiple drafts more frequently (at all ranks and in all tenure groups), but the use of 
student presentations and student evaluations of each other’s work was at about the same rate 
over the decade. Both male and female faculty reported higher use of multiple drafts in 2004 
compared to 1993, however across all three measures female faculty reported high use of all 
assessment techniques. By time status, full-time faculty reported higher use of all three 
techniques in 2004. By type of institution, the use of multiple drafts increased across all 
institution types, and oral presentations and student evaluation of each other’s work remained 
steady. 

To further explore factors related to the use of leaner-c entered assessment techniques, we 
constructed two 2-level hierarchical linear models (HLM) one for 1993 and one for 2004. 
Following the variable construction used by Yanowitz and Hahs-Vaughn (2007), the dependent 
variable in each model was a composite variable representing a participant’s overall use of 
learner-centered assessment techniques. This score was created by adding together a participant’s 
scores on each of the three or five learner-centered techniques (not used at all=l; used in some 
classes=2; used in all classes=3). Tables 3 and 4 presents descriptive statistics for the variables 
entered in each HLM model. 

(Insert Tables 3 and 4 about here) 

Typically, the researcher begins an HLM model with a null or unconditional model, a 
dependent variable, and no other variables. The null model is shown as: 

Level 1 Model 

Yii = Po + nj 
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Level 2 Model 

Po/=Yoo+ uoj ~N(0 , x oo) 

where xoo is the grand mean of learning assessment techniques across institutions and Too is the 
variance between institutions. 

For the 2004 model, the unconditional model for total learner-centered assessment estimated a 
grand mean sum of 7.833 (SE=0. 23064, p <.001). For the 1993 model, the unconditional model 
for total learner-centered assessment estimated a grand mean sum of 4.960 (SE=0. 02979, p 
<.00 1). For both models, the significant chi-square statistic indicated a non-saturated model 
indicated the need to include more variables, thus a broader HLM model was then developed to 
examine the effect of additional factors on learner-centered assessment. The full models with a 
random intercept and fixed slopes are similarly expressed as follows: 

Level 1 Model 

Y(LC Sum) =Po+Pi(Time Status=Full-Time) +P 2 (Femaie) +Ps(Rank=Full 
Professor) +p 4 (Tenure/Tenure-Track) + Pf Would choose Academic Career Again) +PfHAL 
discipline) + py(HAN) + p 8 (HPL)+ p 9 (HPN)+ p,o(SAL)+ pio(SPL)+ p 10 (SPN)+ r 
Level 2 Model 

Po=yoo+ yoi(Ratio research expenditures/totai)+ yo 2 (Mean Faculty-Student Ratio) + yoAPubiic)+ 
yofComprehensive/Master ’s-Gr anting) + yoA Bachelor ’s-Granting/LiberalArts)+ yo6(Associate ’s- 
granting)+ uo 

The continuous variables (FTE student to faculty ratio and ratio of an institution’s total 
expenditures devoted to instruction divided by total E&G expenditures) were grand mean 
centered; dichotomous and categorical variables were not centered. Missing data were removed 
for each analysis as needed, and there were no outliers present in the data. 
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Table 5 presents the results for both hierarchical models. As shown, in both years a 
number of individual and institutional characteristics significantly affected the use of learner- 
centered assessment techniques. In the model for 1993 responses, results for individual level 
characteristics demonstrated significant differences in use by gender, rank, discipline, and 
satisfaction with academic career. Female faculty in 1993 employed learner-centered assessment 
techniques more than male faculty, increasing the grand mean score from 5.502 to 5.928. 
However, full professors used the techniques less; the grand mean score decreased 0.103 to 
5.399. Compared to the referent group in the Soft Pure NonLife discipline, 1993 faculty in all 
other discipline groups used the three assessment techniques significant less. The largest 
deviations from the referent group were generally in the Hard disciplines (generally sciences and 
engineering). 

(Insert Table 5 about here) 

An important benefit to the use of the multilevel model is that it holds constant individual 
characteristics (level one) to allow for precise examination of the influence of institutional 
characteristics on assessment practices. In the 1993 model, differences in the use of assessment 
techniques were not found by ratio of FTE students to faculty, control (public/private) or ratio of 
expenditures apportioned to instructional activities. There were, however, significant differences 
by level of institution. Compared to the reference group, doctoral-granting institutions, faculty at 
comprehensive and liberal arts colleges reported significantly higher use of learner assessment 
techniques. Faculty members in liberal arts institutions reported the highest use with a mean 
score of 5.887. 

Results for the 2004 model shows that some individual and institutional characteristics 
contributed to the use of learner-centered assessment. Similar to 1993, female faculty employed 
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learning-centered assessment techniques to measure student learning more than male faculty, as 
being female increased the grand mean score from 8.738 to 8.596. At the same time, full 
professors made use of these techniques less than faculty in other ra nk s. The grand mean scored 
decreased from 8.738 to 8.533 for full professors. Faculty members in three of the four hard 
disciplines (Hard Applied Non-life, Hard Pure Life, and Hard Pure Non-life) used learner- 
centered techniques significantly less than faculty in the referent group, Soft Pure Non-life 
discipline. Coefficients for three Soft disciplines (shown in Table 5) did not reach significance, 
thus it is appropriate to say that faculty in Hard Applied Life and the three soft disciplines were 
not significantly different in the use of techniques compared to the reference group, Soft Pure 
Non-life. 

In the 2004 model, all institutional variables except FTE student to faculty ratio were 
significant. Faculty in public institutions reported using learner-centered techniques more 
frequently than faculty in private institutions. Compared to doctoral-granting institutions and 
holding all other variables constant, faculty in master’s and bachelor’s level institutions 
employed these techniques more often. For example, being at a master’s-granting institution 
resulted in an increase of 0.369 over the grand mean score of 8.738, and being at a bachelor’ s- 
granting institution resulted in an increase of 0.3 15 over the grand mean score. 

Perhaps one of the most curious findings from the 2004 HLM analysis, faculty in 
institutions that had a higher ratio of expenditures apportioned to instruction employed learner- 
centered assessment techniques significantly less. Holding constant all other variables in the 
model, the grand mean score for dependent variable decreased by 1.710 with each standard 
deviation increase in the ratio of expenditures apportioned to instruction. Perhaps there are wide 
variations in expenses apportioned to instruction for activities such as distance learning. Initial 
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correlation analyses indicated no underlying relationships, thus we are not certain, but consider 
there might be variations in the activities included in instructional expenses that led to the 
negative coefficient. 

Discussion 

Noting the increased calls for continued assessment of student learning, this study 
attempted to examine current use of learner-centered assessment techniques in US colleges and 
universities, as well as compare reported use from 1993 to 2004. Overall, results from this study 
indicate that many of today’s postsecondary faculty are using at least some of the assessment 
techniques considered to be best -practice in terms of learner-centered instruction. In addition, 
results examined herein show that general rate of learner-centered assessment in 2004 is at 
similar, and in some cases, higher rates than reported in 1993. 

In both the last decade of the twentieth century as well as in the first decade of the new 
millennium, over 50% of all faculty members in both NSOPF surveys reported using oral 
presentations in at least some of their classes, and three times as many 2004 faculty used 
multiple drafts in all classes compared to 1993 respondents. In general, faculty in the soft 
disciplines (arts & humanities, education, and social sciences) reported higher use of the 
assessment techniques compared to hard disciplines (physical and life sciences, and 
engineering). This finding is consistent with differences in use of deep learning approaches and 
significant differences found by discipline (Nelson Laird & Garver, 2010). In addition, changes 
in student enrollments over the decade, changes to the undergraduate curriculum, and increasing 
discussion of learner-centered instruction and assessment may affect these percentages over the 
decade. 
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In 2004, two new assessment techniques were included in the survey; over 50% of the 
respondents said they used group or team projects in at least some classes and about 25% said 
they used service-learning /co-op projects. While we acknowledge the 25% use of service- 
learning in at least some classes, it is the least used assessment technique. This lower use may be 
due to the specific courses being offered during the time the survey was completed, faculty lack 
of familiarity with how to incorporate, and/or institutional practices (or lack of) for service 
activities. 

The use of assessment techniques over the 1 1 years from 1993 to 2004 came at a time when we 
have experienced changes in faculty demographics and economic declines that might affect the 
amount of time apportioned to various work tasks, expectations for teaching versus research, 
institutional decisions on resource allocation, and perhaps an indication of some level of 
institutional isomorphism. Compared to 1993 figures, higher percentages of 2004 respondents 
were female, part-time, and employed on no-tenure lines. Consistent with previous reports (e.g., 
Laird Nelson, Garver, & Niskode-Dossett ,2011 and Centra & Gaubatz, 2000), our findings 
show that female use learner-centered assessment techniques more than male peers. Similar to 
Umbach (2008) current findings show that full-time faculty members use these assessment 
techniques more frequently than part-time peers. It is possible that changes in the use of learner- 
centered assessment by gender and time status may be affected by shifting demographics of 
faculty. For example, part-time faculty members may not feel as committed to putting forth extra 
efforts for instruction and/or receive fewer invitations for faculty development that emphasize 
these innovations. 

Lower state appropriations in the new millennium may have forced institutions to cover a 
higher percentage of costs for instruction through other means, and may have resulted in reduced 
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resources allocated for instruction (such as computers and/or classroom renovations for small 
group discussions, a move to hire more part-time instructors, and/or more faculty in no-tenure 
lines. As competition for students and star faculty continue, institution officials seek to 
incorporate policies and practices that attract prospects. For some institutions, the strategy has 
been to market itself uniquely, while for others it is to engage in activities accomplished by 
aspirants. When this happens, it can lead to an institution that looks more like many others than a 
unique institution (perhaps affirming Powell and DiMaggio’s (1991) assertion of institutional 
isomorphism). 

Based on chi-square analyses, significant differences in use of learner-centered 
assessment were found in the proportion of use by gender, rank, discipline, time status, faculty to 
student ratio, and Carnegie classification (Table 2) and many of these findings were confirmed in 
the HLM analyses (Table 5). The finding that tenure and time status were not significant 
contributors in the hierarchical model may have to do with interactions between rank, tenure, and 
time status. Because the proportion of faculty using these techniques was significantly different 
by time and tenure status, the researchers elected to include these variables in the HLM model. 
However, it is possible that the effect for these variables was partially accounted for by rank, 
another closely related variable. Additional study of these variables, including interaction effects, 
is warranted. Differences by discipline are also noted and generally show that instructors who 
teach humanities and education courses use these assessment techniques more than faculty in 
other fields. This difference by discipline, however, could be affected by changes in majors and 
related enrollments in particular courses as well as changes in the composition of faculty (to 
more part-time, female, and those on no-tenure lines). Additional detailed analyses for the use of 



AIR 201 1 Forum, Toronto, Ontario, Canada 




Learner-Centered Assessment 26 



assessment techniques by demographic characteristics within the institution level are warranted, 
as well as additional data that can match assessment techniques to actual learning. 

Although it would disconfinn a hypothesis for institutional isopmorphism across all 
Carnegie types, it is not fully surprising that faculty at bachelor’s- and master’s-granting 
institutions reported the highest use of these techniques. Typically, the missions of bachelor’s 
and master’s level institutions are more focused on teaching, and, therefore, the researchers 
would expect to find greater emphasis and reward for progressive teaching and assessment 
practices. With a focus on vocational training, associate-granting institutions may be less likely 
to adopt learner-centered assessment practices and more likely to emphasize students’ 
comprehension of factual knowledge. While some scholars such as Ewell (2002) argue in the 
value of learner-centered assessment at all levels of postsecondary education, we found evidence 
of similarity for bachelor’s/liberal arts and master’s/comprehensive institutions in their use of 
assessment practices, but differences for these institutions compared to doctoral- and associate- 
granting institutions. This difference might indicate that similarity in institutional accountability 
is not needed or yet strong enough to be seen across all institution types perhaps due to 
differences in institution mission. 

Larger institutions and/or those with a higher student to faculty ratio have relatively more 
resources (both facilities and operating dollars). More resources may include faculty 
development and training that facilitate learner-centered education. However, some smaller 
colleges that specifically seek a high student focus may have activities and instructional 
innovations that facilitate learner-centered assessment that are embedded in the institution’s 
culture. In the multilevel model for 2004 (Table 5) the ratio of expenses apportioned to 
instruction over total E&G expenditures is significant but negative. We acknowledge that 
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differences may be due to activities such as one-time expenses for technology related to 
instruction or a heavy emphasis on distance learning, yet still find this perplexing. Although 
differences in institutional apportionments to instruction would e counter to the concept of 
isomorphism, we thought that institution officials who allocate more funds to instruction would 
have more faculty who use these techniques frequently. Additional funds, we thought, would 
prompt the inclusion of faculty workshops and other training that would contribute to an 
enlivened culture of innovative instruction and assessment for learning. We do not know that 
faculty training did not occur; perhaps it did, but is not part of the available data. Since the effect 
for instructional expenses is nonsignificant for 1993, the negative effect for 2004 may be due a 
one-time result, or differences in activities such as face-to-face and distance learning. Results in 
Tables 2 and 5 show significant differences in the use of the assessment techniques across 
Carnegie type; additional study of the role of resources apportioned to instruction is merited. 
Perhaps our use of the instructional expense ratio is not detailed enough; for example we do not 
what portion of instructional funds are being committed for instructional salaries versus 
classroom renovation, versus expenses for assessment. 

Limitations 

While we believe the use of the learner-centered assessment is an effective gauge that can 
measure authentic learning, the use of these techniques alone does not ensure high levels of 
student learning. Data in the NSOPF surveys do not include measures of learning, so it is not 
possible to know from this dataset the true relationship between use of assessment techniques 
and actual learning outcomes. Data that directly links instructional pedagogy, assessment 
techniques and discreet learning outcomes is needed, but we know of no multi-institution or 
national data that is available for study. 
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Findings presented in the multilevel model are only for faculty who apportion at least 
50% of their time to instruction and do not generalize to others who teach at a lower percentage 
of time. The decision to select this subsample may miss high quality teaching and assessment 
that may occur with individuals who teach occasionally or graduate only courses. Grouping the 
research disciplines into the established Biglan categories may mask effects of unique disciplines 
(or innovative contributions that occur in subdisciplines such as medical school instruction), In 
addition. The Biglan grouping schema does not take into account interdisciplinary teaching. 
However, Biglan’ s classification of academic disciplines has been tested and confirmed as a 
valid method for grouping academic disciplines by several, including Smart and Elton (1982) 
and Stoecker (1993) and the eight categories presented herein are thought to provide a good 
balance of eight categories that allow for distinction across two and four-year colleges yet bring 
together discipline groups that share intellectual content. 

These findings examine assessment practices over a two-year period, and there are other 
facets of learning-centered assessment (e.g., measure of student and/or instructor motivation and 
direct involvement in curriculum development) that are not included in our analyses. 
Achievement motivation theory (Atkinson & Feather, 1966) posits a positive relationship 
between achievement motivation, behaviors, and outcome. Perhaps highly motivated faculty will 
seek to develop more learner-centered activities and assessments. As well, highly motivated 
students might respond more positively to learner-centered instruction, which might in turn, 
prompt or reinforce faculty to develop more learner-centered assessments. 

It is also acknowledged that all data is captured from a self-report survey. In general, 
however, self-report data is purported to be reasonably accurate (Tourangeau, Rips, & Rasinski, 
2000). While analyses presented in the multilevel model represent a number of important 
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individual and institutional characteristics that may contribute to learning-centered assessment, 
an important next step would be to include additional interaction effects. Not included here 
presents a limitation in findings, and a call to include interactions among variables such as 
gender, rank, time status, and discipline in future studies. 

Implications and Recommendations 

It is clear from this body of research that learner-centered assessment is considered a 
valued practice in higher education pedagogy. Findings from this study confirm the stable to 
increased use of learner-centered assessment techniques in higher education in the new 
millennium and reveal that many faculty members have embraced learner-centered assessment 
techniques as an effective way to measure and promote student learning. Findings lead us to 
believe that the larger assessment movement in higher education has taken hold in the classroom 
as well, and the use of these assessment techniques can be one effective piece of an institution’s 
effectiveness plan. The increased use of learner-centered assessment may indicate some 
institutional isomorphism in that accountability agencies are seeking evidence of learning, and 
may be comfortable when familiar techniques are used across institutions. While we see 
evidence of assessment of learning in all levels of postsecondary education, the extent to which 
faculty use these techniques varies by institution, faculty type, and assessment technique. For 
example, faculty appears less likely to use peer evaluation and service-learning/co-op 
opportunities. This may be due to less familiarity with these techniques and/or more resources 
(time and money, in particular) needed to implement them. However, a growing body of 
literature propels service-learning as a highly effective pedagogical technique. Smith (2008), for 
example distinguished it as, “one of the most pervasive education innovations of the past 
generation” (p. 5). Vogelgesang and Astin (2000) described it as a, “potentially powerful form of 
pedagogy,” highlighting the value of students connecting content learned in the classroom to 
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“real world” problems (p. 25). These quotes capture the widely-held belief that infusing service- 
learning into traditional curricula can improve student learning outcomes. In addition, it would 
be helpful to explore the effect of residential living on these techniques, service learning in 
particular. It is plausible that students who live on campus might believe they have more time for 
or proximity to service-learning activities, compared to students who live off campus. To ensure 
continued and increased use of these techniques, senior administrators might showcase existing 
good practice as well as consider the allocation of resources for workshops or other training 
seminars to help faculty become familiar with these techniques. In addition, skilled professionals 
in offices of teaching and learning should be called upon to lead or facilitate such workshops. 
These professionals are often quite knowledgeable of proven and best practice activities related 
to teaching and learning. 

In addition to contributing to the success of faculty, these findings have implications for 
student success. If these techniques promote more effective learning, and if students become 
aware of differences based on the use of these techniques, students may wish to enroll in courses 
that include these assessment activities. The strong increase in multiple drafts may indicate that 
students learn best in successive approximation. Multiple drafts may allow students to focus on 
one or two points at a time, gradually incorporating pieces of knowledge or more advanced 
writing skills. By using these techniques more often, female faculty may develop a more 
interactive, friendly environment in the classroom. Because a positive environment and good 
rapport between instructor and students encourages student engagement (Angelo & Cross, 1993; 
Kuh, 1996; McKeachie, Lin, & Mann, 1971), institutional leaders may wish to encourage the use 
of these assessment techniques, perhaps through faculty workshops or other additional 
expenditures apportioned to instruction. If, however, more female faculty members are using 
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these techniques than male peers, institutional leaders may wish to emphasize the benefits of 
these techniques to male faculty and monitor enrollments by faculty gender to ensure even 
enrollments for male and female faculty. 

The fact that full-time faculty are using these techniques more than part-time faculty may 
indicate greater dedication to instruction. Another explanation is that full-time faculty have more 
time to prepare for instruction generally and assessment activities in particular. Full-time faculty 
may also have more opportunities to participate in professional development opportunities, such 
as training workshops and discussions with other faculty on innovations in teaching and learning. 
As a result, full-time faculty may also be more motivated to integrate progressive practices in 
their classes. As institution officials employ more part-time faculty during economic downturns, 
caution must be taken and to ensure that the quality of instruction and assessment of learning 
does not decrease. Clear messages on the importance of effective assessment as well as faculty 
training to provide needed infonnation and guidance on how to ensure effective pedagogy and 
learner-centered assessment may be required. 

Interestingly, results demonstrated that tenured faculty members use these assessment 
techniques less than non-tenured faculty. Consistent with that reported by Kezar and Sam (2010) 
this finding may be due to the fact that non-tenured faculty are younger in their career, recently 
our of graduate school and/or more focused on instruction (while full professors may be more 
engaged in research), and/or could also indicate that non-tenured faculty members are working in 
institutions that have a stronger focus on instruction. These institutions may provide training in 
learner-centered assessment or otherwise encourage the use of these assessment techniques. If 
institutional leaders desire higher use of these activities, clear messages on the value of learner- 
centered assessment as well as faculty training should be put in place. 
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Differences in frequency of use by discipline may be the result of different pedagogies, 
topical content, and/or cultural differences by department. For example, faculty who teach 
courses with more hands-on applications of content may find it more natural to integrate learner- 
centered techniques in their courses. It is noteworthy that in general, faculty members in many of 
the hard disciplines report using these techniques less than peers in the soft disciplines. While 
medical school and some biology curricula have used problem-based learning for at least two 
decades (Wood, 2004) and biology (Duch, Groh & Allen, 2001), some other science disciplines 
may not be incorporating this or other pedagogies that include learner-centered assessment. 
Institution officials may wish to showcase the innovations of select faculty who can serve as peer 
mentors and role models to other faculty. To help propel the message, institutions officials may 
wish to provide a small summer stipend for faculty members who incorporate new assessment 
techniques in their courses or designate a learner-centered instructor of the year. 

Although we acknowledge the steady, and in some cases, increased use of some learner- 
centered assessment techniques from 1993 to 2004, we also note that some faculty members 
chose to not incorporate these techniques into any of their courses. Of note, over 60% of the 
respondents said they did not use student’s evaluations of each other’s work and nearly % of the 
2004 respondents did not use service-learning, co-op or other community-related activities is any 
of their courses. Although it is unclear from the data, perhaps faculty members believe peer 
feedback is less helpful than that given by the instructor. Or perhaps the need to apportion time 
across multiple tasks encourages faculty to complete assessments that are quick and easy. While 
we understand the challenges that may exist related to resources and determining activities that 
assist in authentic learning, we also acknowledge the long-tenn value for both faculty and 
students when this occurs. For example, academic courses with a group/team or service-learning 
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component are critical and effective ways to enable students to cognitively understand the 
connection between themselves and others in society, one of the enduring goals of postsecondary 
education. 
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Tablet. Descriptive Statistics 



Variable Label 


1993 


2004 


N* 


% 


N* 


% 


Gender 










Male 


9220 


61.1 


9340 


57.5 


Female 


5870 


38.9 


6900 


42.5 


Position type 










Full-time 


9110 


60.4 


8550 


55.6 


Part-time 


5980 


39.6 


7700 


47.4 


Rank 










Full Professor 


3280 


22.4 


2770 


17.5 


Associate Professor 


2490 


17.0 


2130 


13.5 


Assistant Professor 


2580 


17.7 


2330 


14.7 


Instructor 


4890 


33.4 


3960 


25.0 


Lecturer 


980 


6.7 


1020 


6.4 


Other title 


400 


2.6 


3620 


22.9 


Tenure Status 










Tenured 


5440 


36.1 


4630 


28.5 


On tenure track but not tenured 


2170 


14.4 


2130 


13.1 


Not on tenure track 


3430 


22.7 


8110 


49.9 


No tenure system at institution 


980 


6.5 


1380 


8.5 


No tenure for my faculty status 


3060 


20.3 


NA 




Discipline 










Hard Applied Life 


2190 


14.5 


1550 


9.6 


Hard Applied Non-Life 


1060 


7.0 


1680 


10.4 


Hard Pure Life 


620 


4.1 


700 


4.3 


Hard Pure Non-life 


1620 


10.7 


1670 


10.3 


Soft Applied Life 


2640 


17.5 


3860 


23.8 


Soft Applied Non-life 


2020 


13.4 


1630 


10.1 


Soft Pure Life 


1040 


6.9 


1370 


8.5 


Soft Pure Non-life 


3100 


20.5 


3720 


23.0 


Institutional control 










Public 


580 


72.5 


390 


63.3 


Private 


220 


27.5 


220 


36.7 


Carnegie classification 










Doctoral-granting institutions 


180 


23.0 


140 


23.5 


Comprehensive/Master’s universities and colleges 


220 


27.2 


130 


22.8 


Liberal Arts/Baccalaureate Colleges 


70 


8.6 


110 


19.0 


Associates-granting institutions 


330 


41.1 


200 


34.7 
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Table 1, continued. 
Descriptive Statistics 





1993 




2004 




Variable Label 


N* 


% 


N* 


% 


Learner-centered assessment techniques 

Multiple drafts of written work 










Used in all classes 


1650 


13.5 


5610 


41.1 


Used in some classes 


2360 


19.3 


2290 


16.9 


Not used at all 


8200 


67.2 


5650 


41.7 


Oral presentations by students 










Used in all classes 


3080 


25.3 


4950 


36.5 


Used in some classes 


4700 


38.5 


3250 


24.0 


Not used at all 


4430 


36.3 


5360 


39.6 


Student evaluations of each other’s work 










Used in all classes 


1710 


14.0 


2860 


21.1 


Used in some classes 


2900 


23.7 


2180 


16.1 


Not used at all 


7600 


62.3 


8510 


62.8 


Group and team projects producing a joint product 










Used in all classes 


N/A 




4290 


31.7 


Used in some classes 


N/A 




2980 


22.0 


Not used at all 


N/A 




8510 


46.4 


Service-learning, co-op experiences or assignments 
requiring interactions with the community or 
business/industry 










Used in all classes 


N/A 




1720 


12.7 


Used in some classes 


N/A 




1900 


14.0 


Not used at all 


N/A 




9940 


73.3 


TA in one or more class(es) 










Yes 


2190 


14.5 


2970 


18.3 


No 


12900 


85.5 


12480 


81.0 



*A11 Ns are weighted and rounded 
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Table 2. Proportion of Facu 


Ity Using Learning-centered Assessment in 


at Least Some Classes by Demographic Characteristics and Discipline 




1993* 


2004** 


Variable Label 


Multiple 


Oral 


Student 


Multiple 


Oral 


Student 


Group 


Service- 




drafts 


presentations 


evaluations 


drafts 


presentations 


evaluations 


projects 


learning 








each other’s 






each other’s 




assignments 








work 






work 






Rank 


















Full professor 


33.2 


67.5 


33.2 


43.8 


62.7 


33.4 


51.4 


23.3 


Associate professor 


34.4 


65.1 


37.5 


45.4 


63.1 


38.1 


58.1 


28.5 


Assistant professor 


37.2 


68.8 


42.0 


46.7 


67.1 


41.9 


59.5 


29.3 


Instructor 


29.3 


60.8 


37.4 


36.0 


58.2 


38.9 


53.3 


29.6 


Lecturer 


34.9 


65.0 


42.8 


40.2 


62.3 


41.3 


53.7 


23.4 


Tenure Status 


















Tenured 


33.3 


63.3 


34.8 


44.5 


62.1 


35.1 


54.6 


25.9 


On tenure track but not 


38.5 


68.4 


42.4 


47.2 


68.6 


44.2 


63.0 


32.2 


tenured 

Not on tenure track 


31.3 


64.5 


40.5 


33.6 


57.6 


36.9 


52.2 


24.4 


No tenure system at 
institution 


31.5 


66.2 


35.4 


32.9 


57.9 


37.2 


54.9 


33.3 


Gender 


















Male 


28.2 


58.8 


31.6 


34.1 


54.6 


30.2 


49.1 


21.3 


Female 


39.9 


71.3 


47.1 


44.7 


68.5 


46.8 


59.8 


34.1 


Discipline 


















Flard Applied Life 


25.2 


67.9 


34.1 


28.0 


61.8 


34.4 


56.0 


47.7 


Hard Applied Non-life 


18.3 


45.2 


21.7 


24.6 


52.4 


31.8 


59.8 


22.8 


Hard Pure Life 


24.3 


48.7 


24.3 


31.6 


49.5 


25.3 


48.4 


12.1 


Hard Pure Non-life 


12.1 


33.7 


14.3 


19.0 


31.1 


15.9 


40.1 


8.6 


Soft Applied Life 


32.4 


78.6 


56.7 


36.5 


69.4 


47.6 


56.7 


37.5 


Soft Applied Non-life 


22.5 


59.4 


27.3 


33.7 


63.8 


34.9 


63.5 


29.0 


Soft Pure Life 


38.3 


72.2 


29.8 


49.4 


73.9 


34.0 


53.4 


27.6 


Sort Pure Non-life 


58.8 


72.9 


53.4 


58.1 


64.5 


44.9 


50.9 


21.6 
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Table 2, continued 







1993* 








2004** 






Variable Label 


Multiple 

drafts 


Oral 

presentations 


Student 
evaluations 
each other’s 
work 


Multiple 

drafts 


Oral 

presentations 


Student 
evaluations 
each other’s 
work 


Group 

projects 


Service- 

learning 

assignments 


Position type 


















Full-time 


35.3 


65.5 


37.7 


44.5 


39.3 


39.3 


58.3 


29.3 


Part-Time 


28.7 


60.8 


37.7 


31.5 


34.8 


34.8 


48.1 


23.4 


Carnegie Classification 


















Doctoral-granting 


31.8 


59.1 


34.1 


39.1 


34.46 


34.4 


49.9 


21.6 


institutions 
Master’s universities 


35.9 


68.9 


40.3 


43.0 


40.5 


40.5 


58.3 


28.9 


and colleges 
Baccalaureate colleges 


43.1 


78.3 


44.5 


49.7 


41.9 


41.9 


59.8 


28.4 


Associate’s colleges 


28.6 


59.0 


36.2 


32.6 


35.7 


35.7 


51.3 


28.0 



*all chi-square values were significant except oral presentations by time status 

**all chi-square values were significant 

all percentages are based on weighted scores 
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Table 3. Descriptive Statistics for NSOPF: 1993 Variables in The HLM Model 



Variable label 


N* 


Min. 


Max. 


Mean. 


SD 


Rank 

(l=full; 0=other) 


15300 


.00 


1.00 


.59 


.491 


Tenure and Tenure -track 
(l=tenure; 0=other) 


15300 


.00 


1.00 


.505 


.500 


Female 


15300 


.00 


1.00 


.389 


.487 


Discipline 












Flard Applied Life 


15300 


.00 


1.00 


.145 


.352 


Flard Applied Non-life 


15300 


.00 


1.00 


.069 


.255 


Flard Pure Life 


15300 


.00 


1.00 


.041 


.199 


Flard Pure Non-life 


15300 


.00 


1.00 


.103 


.309 


Soft Applied Life 


15300 


.00 


1.00 


.175 


.380 


Soft Applied Non-life 


15300 


.00 


1.00 


.128 


.341 


Soft Pure Life 


15300 


.00 


1.00 


.081 


.275 


Soft Pure Non-life 


15300 


.00 


1.00 


.241 


.431 


Position type 


15300 


.00 


1.00 


0.74 


.443 


(l=fiill-time; 0=other) 












Faculty student ratio 


680 


0.4 


77.6 


18.39 


10.412 


Carnegie classification 












Doctoral-granting 


680 


.00 


1.00 


.248 


.428 


Master’s universities and 


680 


.00 


1.00 


.277 


.445 


colleges 
Liberal Arts/ 


680 


.00 


1.00 


.091 


.281 


Baccalaureate Colleges 
Associate’s Colleges 


680 


.00 


1.00 


.393 


.492 


% of total expenditures 


680 


.041 


.723 


.438 


.723 


devoted to instruction 












Institutional control 


680 


0 


1 


.725 


.447 



(l=public; 0=other) 



*A11 Ns are rounded 
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Table 4. Descriptive Statistics for NSOPF: 2004 Variables in The HLM Model 



Variable label 


N* 


Min. 


Max. 


Mean. 


SD 


Rank 

(l=full; 0=other) 


10280 


.00 


1.00 


.192 


.390 


Tenure and Tenure -track 
(l=tenure; 0=other) 


10550 


.00 


1.00 


.501 


.502 


Female 


10550 


.00 


1.00 


.452 


.496 


Discipline 












Flard Applied Life 


10550 


.00 


1.00 


.095 


.281 


Flard Applied Non-life 


10550 


.00 


1.00 


.093 


.294 


Flard Pure Life 


10550 


.00 


1.00 


.048 


.208 


Flard Pure Non-life 


10550 


.00 


1.00 


.109 


.312 


Soft Applied Life 


10550 


.00 


1.00 


.237 


.425 


Soft Applied Non-life 


10550 


.00 


1.00 


.091 


.289 


Soft Pure Life 


10550 


.00 


1.00 


.101 


.301 


Soft Pure Non-life 


10550 


.00 


1.00 


.239 


.429 


Position type 


16250 


.00 


1.00 


.5263 


.49932 


(l=fiill-time; 0=other) 












Faculty student ratio 


550 


1 


86.00 


15.31 


5.78 


Carnegie classification 












Doctoral-granting 


550 


.00 


1.00 


.241 


.426 


Master’s universities and 


550 


.00 


1.00 


.227 


.421 


colleges 

Baccalaureate Colleges 


550 


.00 


1.00 


.187 


.388 


Associate’s Colleges 


550 


.00 


1.00 


.336 


.469 


% of total expenditures 


550 


.01 


.704 


.416 


.087 


devoted to instruction 












Institutional control 


550 


0 


1 


.662 


.475 



(l=public; 0=other) 



*A11 Ns are rounded 
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Table 5. HLM Results for the 1993 and 2004 Models 





1993 

(range of score 3-9) 


2004 

(range of score 5-15) 




P 

Coefficient 


Sig 


P 

Coefficient 


Sig 


Intercept 
Level 2 


5.502 


** 


8.738 


** 


FTE Student-FTE Faculty Ratio 


0.001 




0.012 




Ratio instruction expenditures to total 


0.012 




-1.710 


* 


Control (l=public, 0=private) 


-0.153 




0.394 


** 


Carnegie2-Master’s-granting/Comprehenisive a 


0.275 


** 


0.315 


* 


Carnegie 3-Bachelor’s-granting/Liberal Arts 


0.385 


* 


0.369 


* 


Carnegie4-Associates-granting 


0.069 




-0.059 




Level 1 










Female (l=yes, 0=no) 


0.426 


** 


0.858 


** 


Rank- full professor (l=yes, 0=all others) 


-0.103 


* 


-0.205 


* 


Tenure (l=yes, 0=no) 


0.034 




0.140 




Time status (1=FT, 0=PT) 


-0.043 




-0.075 




Would choose academic job again 


-0.067 


* 


-0.039 




Hard- Applied-Life Discipline b 


-0.917 


** 


-0.300 




Hard Applied Nonlife 


-1.120 


** 


-0.779 


** 


Hard Pure Life 


-1.176 


** 


-0.912 


** 


Hard Pure Nonlife 


-1.609 


** 


-2.000 


** 


Soft Applied Life 


-0.0207 


** 


0.163 




Soft Applied Nonlife 


-0.933 


** 


-0.084 




Soft Pure Life 


-0.655 


** 


-0.091 




Variance Components 










Level 1 o2 


2.214 




5.872 




Level 2 too 


0.0597 


** 


0.282 


** 


ICC 


.0267 




.0458 





*p< 05; **p<.0l 

a referent group is doctoral-granting institutions 
b referent group is Soft Pure Non-life discipline 
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Appendix A 

NSOPF Teaching Field Variables to Biglan Categories 


Biglan Category 


NSOPF Teaching Field 


Hard-Applied-Life 

(HAL) 


Agriculture/natural resources/related 
Construction trades 
Health professions/clinical sciences 
Transportation and materials moving 


Hard-Applied-Non-life 

(HAN) 


Architecture and related services 
Computer/info sciences/support tech 
Engineering technologies/technician 
MechanicaPrepair technologies/techs 
Science technologies/technicians 
Transportation & materials moving 


Hard-Pure -Life 
(HPL) 


Biological and biomedical sciences 


Hard-Pure -Non-life 
(HPN) 


Mathematics and Statistics 
Precision Production 
Physical Sciences 


Soft -Applied-Life 
(SAL) 


Visual and Performing Arts 

Business/management/marketing/related 

Economics 

Education 

Multidisciplinary studies 
Parks/recreation/leisure/fitness studies 
Personal and culinary services 
Public administration/social services 
Security and protective services 


Soft -Applied Non-life 
(SAN) 


Legal professions and studies 


Soft-Pure -Life 
(SPL) 


Family/consumer science, human sciences 
F oreign languages/literature/linguistics 
Psychology 

Area/ethnic/ cultural/ gender studies 


Soft-Pure -Non-life 
(SPN) 


C ommunity/j oumalism/ community tech 
English languages and literatures 
Philosophy, religion and theology 
Library science 

Social sciences (except psych) and history 
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